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Background / Context: 

Description of prior research and its intellectual context. 


The 1998 reauthorization of Head Start called for a national evaluation of the Head Start 
program. The goal of Head Start is to improve the school readiness skills of low-income 
children. Yet characteristics of Head Start programs, such as their quality and the amount of 
time children spend in them may influence their effectiveness at achieving this goal. Previous 
research has demonstrated that children who spend more weekly hours in Head Start demonstrate 
larger cognitive gains than children who attend fewer weekly hours (Li, Farkas, Duncan, 

Vandell, & Burchinal, 2013). But the quality of care children attend during such hours may also 
matter, as there is mounting evidence regarding the importance of classroom quality for 
children’s developing school readiness skills (Burchinal, Vandergrift, Pianta, & Mashbum, 2009; 
Zaslow et ah, 2010). Thus, the effect of weekly hours in Head Start may vary by the quality of 
the Head Start classroom. Indeed, previous research has found that child care quality moderates 
the association between hours in child care and child behavior problems, but not cognitive skills 
(McCartney et ah, 2010; Votruba-Drzal, Coley, & Chase-Lansdale, 2004). 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 


The current study expands on previous research by using quasi-experimental methods that 
leverage the experimental context of the Head Start Impact Study (HSIS; Puma et ah, 2010) to 
understand the extent to which Head Start classroom quality moderates the impact of weekly 
hours in Head Start on children’s early language and math skills and externalizing behaviors. 

We begin by replicating Li and colleagues’ (2013) instrumental variables (IV) analysis assessing 
the effects of weekly hours in Head Start on outcomes for children, leveraging the random 
assignment nature of the HSIS design and the “offer” of differing numbers of hours of Head Start 
in the treatment condition and zero hours of Head Start in the control condition. We then extend 
this work to account for the quality of the Head Start center children attend. We hypothesize that 
weekly hours in Head Start will be more strongly associated with outcomes for children enrolled 
in high quality programs as compared with children enrolled in low quality programs. In contrast 
to previous research that used samples of children enrolled in child care, the current study relies 
on a sample of children enrolled in educationally focused Head Start programs. 

Setting: 

Description of the research location. 


The Head Start Impact Study sample was designed to be nationally representative of 3- and 4- 
year-olds attending Head Start programs in the United States and included children in 22 states. 
Observations of classroom quality occurred in the child’s Head Start center. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

This research uses data from the Head Start Impact Study in which 4,440 3- and 4-year-old 
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predominantly low-income children were randomly assigned off a waitlist to either receive an 
invitation to participate in Head Start services or to the control group. Children applied to 35 1 
Head Start centers. Small centers were combined with nearby centers to form 202 center groups 
and children were randomly assigned to Head Start from these center groups rather than the 
Head Start center to which they applied. The sample for these analyses was limited to children 
who had data on at least one of the outcome measures, and to children in one randomly selected 
Head Start random assignment center per center group. We randomly selected one center in 
order that the assessment of quality be reflective of the center to which children applied (rather 
than the average quality across centers in center groups). After including only center groups in 
which there was non-missing data for children in both random assignment groups, the analytic 
sample included 2,482 children in 270 Head Start centers and 697 Head Start classrooms. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 


Children were randomly assigned to Head Start or to a control group. Children assigned to Head 
Start had access to the services that the Head Start program provided, including a certain number 
of hours of center-based care and related medical, dental, nutrition, and mental health services. 
The control group could not enroll in that Head Start center, but could enroll in other early care 
programs (i.e. non-Head Start center based programs and family child care) or were cared for at 
home by a parent. Children in the control group did not have access to Head Start but there were 
some “crossovers” in which children in the control condition attended other Head Start centers. 

Research Design: 

Description of the research design. 


Random assignment occurred prior to the beginning of the 2002-03 school year. Data collection 
began in the fall of 2002, after random assignment and continued through the spring of 2003. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 


The current study utilizes data collected during the first year of the longitudinal Head Start 
Impact Study. Children’s early math and language skills were assessed using the Peabody Picture 
Vocabulary Test (PPVT; Dunn, Dunn, & Dunn, 1997) and Woodcock-Johnson Letter-Word 
Identification and Applied Problems subtests (Woodcock, McGrew, & Mather, 2001) during the 
fall and spring in the child’s home or Head Start center by trained data collectors. The PPVT 
measures children’s receptive vocabulary while the Letter-Word Identification task assesses 
children’s ability to identify and name letters and words. The Applied Problems tasks measures 
children’s ability to analyze and solve math problems. Children’s externalizing problems were 
assessed using a parent report based on the Child Behavior Checklist (Achenbach, Edelbrock, & 
Howel, 1987). Seven items regarding children’s hyperactive and aggressive behaviors were 
combined to form the externalizing problems scale (Alpha = 0.71). In order to improve the 
interpretation of findings, all outcome measures were standardized to z-scores. 

Classroom quality was measured using the Early Childhood Environment Rating Scale (ECERS- 
R; Harms, Clifford, & Cryer, 1998) and the Arnett Caregiver Interaction Scale (CIS; Arnett, 
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1989) during the spring of 2003. The ECERS-R and CIS are observation tools used to measure 
early childhood classroom quality. Exploratory and confirmatory factor analysis were used to 
extract three construct specific factors across these two tools. The resulting construct specific 
factors are: Materials & Space for Eearning, Positive Teacher-Child Interactions, and Negative 
Teacher-Child Interactions (Connors, Eriedman-Krauss, Jones, & Morris, in preparation). In the 
spring of 2003 Head Start center directors reported the daily operating schedule for their center 
which was used to compute the number of weekly hours of Head Start offered. Parents reported 
the number of hours per week their child attended Head Start. 

Our first step was to replicate previous work by Ei and colleagues (2013) which, using an 
instrumental variables approach, found a positive effect of attending more weekly hours of Head 
Start on children’s math and language abilities and behavior problems. IV analyses allow for the 
approximation of a causal estimate by using an “instrument” to isolate the exogenous variance in 
a “mediator” variable (i.e. variance that is not correlated with child or family characteristics) and 
using that exogenous variation to predict the outcome of interest (Gennetian, Morris, Bos, & 
Bloom, 2005). IV analyses reduce the threat of selection bias due to characteristics of children 
and families that are associated with selection into Head Start care settings, quality of those care 
settings, and child outcomes. We used the number of hours per week offered by a Head Start 
center as an instrument; hours of Head Start offered was zero for any child randomized to the 
control group as they were not offered Head Start services. In the first stage we used hours of 
Head Start offered to predict the number of hours a child attended Head Start. In the second stage 
we estimated child outcomes from the predicted hours in Head Start generated from stage 1 : 

( 1 ) y Hours in Head Start = Hio+ HiiHours Offeredn -t-HikCovariateski + XninCenterGroupsni + pn 

(2) Y child outcome = Bo -t BiHours in HS -i- BkCovariateski + XBnCenterGrougSni + Si 

In both stages we included a set of indicators to control for the child’s center group and a limited 
set of baseline covariates (including children’s pre -random assignment outcomes) in order to 
increase the precision of the instrumental variable estimates (Gennetian et ah, 2005). Multiple 
imputation was used to replace missing data on baseline covariates. 

In a first phase of this work, we split the sample into groups of children attending high and low 
quality programs in order to test our hypothesis that weekly hours in Head Start is more strongly 
associated with positive child outcomes in high as compared with low quality centers. Children 
who were not in Head Start classrooms were missing a Head Start classroom quality score. In 
order to group these children into either high or low quality, we used the average Head Start 
classroom quality across all Head Start classrooms attended by children who applied initially to 
the same Head Start center. In doing so, we assume that the quality of children’s early care 
experiences is relatively homogenous within the neighborhood where they applied to Head Start. 
Eor each of the three construct specific quality factors we split the sample into high and low 
quality based on the median of the distribution of average quality. We repeated our analyses 
(equations 1 and 2) separately for each of the high and low quality groups and used t tests to 
compared the impact estimated in the high and low quality subsamples. Euture analyses will 
consider quality and the interaction between the number of hours in Head Start and quality as 
endogenous variables in instrumental variables models, utilizing additional instruments. 


SREE Spring 2014 Conference Abstract Template 


A-3 



Findings / Results: 

Description of the main findings with specific details. 

Results of the IV analysis show that, first and foremost, the F-statistics from the first stage was 
greater than 30 and weekly hours of Head Start offered predicted weekly hours of Head Start 
attended (^7=0.41, S.E.=0.01, p<0.001) indicating sufficient strength in our instrument to isolate 
the exogenous variation in weekly hours of Head Start attended. Table 1 presents the impacts of 
weekly hours in Head Start on children’s development. Children who attended more weekly 
hours of Head Start demonstrated significantly higher math and language skills at the end of the 
school year. Children who attended more weekly hours of Head Start were rated as having lower 
externalizing behaviors but this relationship was not significant (please insert table 1 here). 

Tables 2, 3, and 4 present the results of the separate instrumental variables analyses for the high 
and low quality groups for each construct-specific quality factor. More hours in Head Start 
significantly predicted higher math in high but not low quality Head Start programs when quality 
was measured by Materials & Space or Negative Interactions. These differences in the effect of 
weekly hours in high and low quality Head Start were significant at the trend level. Additionally, 
the effect of weekly hours in Head Start on children’s externalizing behaviors was statistically 
significantly different (at the trend level) in high quality compared to low quality Head Start 
when quality was measured by Materials & Space (please insert tables 2, 3, & 4 here). 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 


The results of this study are in line with previous findings that more hours in Head Start and 
higher quality classrooms are associated with higher school readiness skills. We find that both 
weekly hours in Head Start and the quality of the Head Start program are important, especially 
for children’s math skills. Translating our hourly effects in high quality programs into full-day 
estimates (i.e., 40 hours) amounts to moderate effect sizes (ranging from 0.29 to 0.44) on math 
and language outcomes. These estimated effect sizes are in line with a recent meta-analysis of 
early childhood programs (Yoshikawa et ah, 2013). Moreover, a child enrolled in a high quality, 
as measured by low negative interactions, full-day program is estimated to perform 0.32 standard 
deviations higher in math compared to a child enrolled in a low quality full-day program. 

Notably, our analyses considered quality as exogenous (with the instrumental variables 
methodology only accounting for the endogeneity in weekly hours of Head Start). In future 
analyses, we plan to address this issue by instrumenting hours and quality of Head Start as well 
as their interaction in assessing their effects on outcomes for children. 

These results suggest that program operating schedule and quality should be considered 
simultaneously when thinking about the circumstances under which Head Start has positive 
effects on children. As Head Start and other educationally focused early childhood programs 
expand, it may be that providing more hours is only be beneficial (and therefore cost effective) if 
programs provide and maintain high quality classrooms, and vice versa. The finding that more 
weekly hours in Head Start positively impacts children’s academic skills highlights the tradeoff 
between the costs and benefits of enrolling more children for fewer hours or fewer children for 
more hours. 
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Appendix B. Tables and Figures 
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Table 1. IV estimates of the effects of hours per week of Head Start on child outcomes 



PPVT 

WJ Letter 
Word 

WJ Applied 
Problems 

Externalizing 

Problems 

OLS 

Bi Hours 

0.0024*** 

0.0071*** 

0.0037*** 

-0.0014 

(se) 

(0.0010) 

(0.0012) 

(0.0012) 

(0.0013) 

Covariates 

XXX 

XXX 

XXX 

XXX 

F 

18.82*** 

9 Y7*** 

8.05*** 

6.28*** 

IV 

Bi Hours 

0.0077*** 

0.0104*** 

0.0066*** 

-0.0005 

(se) 

(0.0016) 

(0.0019) 

(0.0020) 

(0.0021) 

Covariates 

XXX 

XXX 

XXX 

XXX 

F 

18.54*** 

9.64*** 

8.02*** 

6.27*** 


Notes: ***/?<0.001. Dependent variables are standardized using z-scores. Control variables include baseline 
assessment of outcome, child gender, if the child was assessed at baseline in Spanish, if the child’s mother had less 
than a high school diploma, an indicator for if the child applied to Head Start as a three- or four-year-old, and a set 
of indicator variables for the center group from which random assignment occurred to account for nesting. 
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Table 2. IV estimates of the effects of hours per week of Head Start on child outcome by high 
and low quality based on Materials & Space for Learning 



PPVT 

WJ Letter 
Word 

WJ Applied 
Problems 

Externalizing 

Problems 

High Quality 

Bi Hours 

0.0077*** 

0.0126*** 

0.0098*** 

-0.0036 

(se) 

(0.0021) 

(0.0027) 

(0.0028) 

(0.0028) 

covariates 

XXX 

XXX 

XXX 

XXX 

F 

19.55*** 

g g4*** 

7 33*** 

6.82*** 

Low Quality 

Bi Hours 

0.0072** 

0.0079** 

0.0023 

0.0035 

(se) 

(0.0023) 

(0.0028) 

(0.0029) 

(0.0031) 

covariates 

XXX 

XXX 

XXX 

XXX 

F 

17 44*** 

10.23*** 

8.68*** 

5.63*** 

P value for Difference 

Between High and Low 

0.87 

0.24 

0.07^' 

0.10+ 

Quality 


Notes: ^/?<0.10, **p<Q.Ql, ***/?<0.001. Dependent variables are standardized using z-scores. Control variables 
include baseline assessment of outcome, child gender, if the child was assessed at baseline in Spanish, if the child’s 
mother had less than a high school diploma, an indicator for if the child applied to Head Start as a three- or four- 
year-old, and a set of indicator variables for the center group from which random assignment occurred to account for 
nesting. 
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Table 3. IV estimates of the effects of hours per week of Head Start on child outcome by high 
and low quality based on Positive Teacher-Child Interactions 



PPVT 

WJ Letter 
Word 

WJ Applied 
Problems 

Externalizing 

Problems 

High Quality 

Bi Hours 

0.0080*** 

0.0122*** 

0.0073* 

-0.0011 

(se) 

(0.0023) 

(0.0028) 

(0.0029) 

(0.0029) 

covariates 

XXX 

XXX 

XXX 

XXX 

F 

16.27*** 

7 54*** 

6.35*** 

5.62*** 

Low Quality 

Bi Hours 

0.0070** 

0.0083** 

0.0047T 

0.0011 

(se) 

(0.0021) 

(0.0027) 

(0.0028) 

(0.0029) 

covariates 

XXX 

XXX 

XXX 

XXX 

F 

21.16*** 

\ 7]^*** 

10.09*** 

5 94*** 

P value for Difference 

Between High and Low 

0.75 

0.32 

0.52 

0.60 

Quality 


Notes: *p<0.05, **p<0.01, ***p<0.001. Dependent variables are standardized using z-scores. Control variables 
include baseline assessment of outcome, child gender, if the child was assessed at baseline in Spanish, if the child’s 
mother had less than a high school diploma, an indicator for if the child applied to Head Start as a three- or four- 
year-old, and a set of indicator variables for the center group from which random assignment occurred to account for 
nesting. 
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Table 4. IV estimates of the effects of hours per week of Head Start on child outcome by high 
and low quality based on Negative Teacher-Child Interactions 



PPVT 

WJ Letter 
Word 

WJ Applied 
Problems 

Externalizing 

Problems 

High Quality 

Bi Hours 

0.0085*** 

0.0079* 

0.0109** 

0.0012 

(se) 

(0.0025) 

(0.0032) 

(0.0034) 

(0.0033) 

covariates 

XXX 

XXX 

XXX 

XXX 

F 

18.43*** 

g 21*** 

6.42*** 

6.89*** 

Low Quality 

Bi Hours 

0.0069*** 

0.0119*** 

0.0029 

-0.0009 

(se) 

(0.0020) 

(0.0024) 

(0.0024) 

(0.0026) 

covariates 

XXX 

XXX 

XXX 

XXX 

F 

18.40*** 

g 22*** 

9 59*** 

5.63*** 

P value for Difference 

Between High and Low 

0.62 

0.33 

0.06^ 

0.62 

Quality 


Notes: '/?<0.10, *p<0.05, **p<0.01, ***p<0.001. Dependent variables are standardized using z-scores. Control 
variables include baseline assessment of outcome, child gender, if the child was assessed at baseline in Spanish, if 
the child’s mother had less than a high school diploma, an indicator for if the child applied to Head Start as a three- 
or four-year-old, and a set of indicator variables for the center group from which random assignment occurred to 
account for nesting. 
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